Skip to content

Include realized build time savings in experiment summary #261

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 19 commits into from
Mar 6, 2023
Merged

Include realized build time savings in experiment summary #261

merged 19 commits into from
Mar 6, 2023

Conversation

erichaagdev
Copy link
Member

@erichaagdev erichaagdev commented Dec 28, 2022 •
edited
Loading

❄️ Winter Update PR Navigator ❄️

  1. ⛄ Include realized build time savings in experiment summary #261
  2. Include performance characteristics and support -x for Gradle experiment 1  #262
  3. Always read project name from Build Scan data #263
  4. Gradle init script improvements #264
  5. Adopt Foojay Toolchains and Kotlin DSL plugins in build logic #265
  6. Add variable to query debug mode #266
  7. Modernize Java launch code #267
  8. Display serialization factor in summary of all experiments #272
  9. Display potential build time savings in experiment summaries #274
  10. Update the contract of command line tools #283

⛄ = you are here

Summary

ℹ️ This PR only partially implements #206. This felt like a good place to stop. More PRs will be coming.

This PR introduces the following changes:

  • The API command line tool now includes Effective task execution duration in the CSV response
  • The Build Caching Leverage section of the experiment summary has been renamed to Performance Characteristics
  • Realized build time savings is now included in the Performance Characteristics section of the experiment summary

Example

Performance Characteristics
----------------------
Realized build time savings:  1m 7.377s wall-clock time (from 1m 11.542s to 4.165s)
Avoided cacheable tasks:      292 tasks, 4m 44.680s total saved execution time
Executed cacheable tasks:       1 tasks, 0.045s total execution time
Executed non-cacheable tasks: 292 tasks, 10.686s total execution time

WARNING: Not all cacheable tasks' outputs were taken from the build cache in the second build. This reduces the savings in task execution time.

Notes

  • I ensured this change was backwards compatible with the current version of the build-scan-support-tool.jar. If the new Effective task execution duration field is not included in the command line CSV response then Realized build time savings will not be displayed.
  • For the display format of Realized build time savings I decided to take @runningcode's suggestion. I even tried a few display variations myself and liked his suggestion the best. Let me know if the format should be changed.

Testing

Gradle

Experiment 1

./01-validate-incremental-building.sh \
  -r git@github.com:gradle/ge-solutions.git \
  -p sample-projects/gradle/7.x/ge \
  -t 'build'

No visual differences.

image

Experiment 2

Online
./02-validate-local-build-caching-same-location.sh \
  -r git@github.com:gradle/ge-solutions.git \
  -p sample-projects/gradle/7.x/ge \
  -t 'build'

image

Offline
./02-validate-local-build-caching-same-location.sh \
  -r git@github.com:gradle/ge-solutions.git \
  -p sample-projects/gradle/7.x/ge \
  -t 'build' \
  -x

No visual differences.

image

Experiment 3

Online
./03-validate-local-build-caching-different-locations.sh \
  -r git@github.com:gradle/ge-solutions.git \
  -p sample-projects/gradle/7.x/ge \
  -t 'build'

image

Offline
./03-validate-local-build-caching-different-locations.sh \
  -r git@github.com:gradle/ge-solutions.git \
  -p sample-projects/gradle/7.x/ge \
  -t 'build' \
  -x

No visual differences.

image

Experiment 4

./04-validate-remote-build-caching-ci-ci.sh \
  -1 https://ge.solutions-team.gradle.com/s/fvwne3ht4xgjg \
  -2 https://ge.solutions-team.gradle.com/s/tjiwbi2b7oioo

image

Experiment 5

./05-validate-remote-build-caching-ci-local.sh \
  -1 https://ge.solutions-team.gradle.com/s/lc73bw56kdkxi

image

Maven

Experiment 1

Online
./01-validate-local-build-caching-same-location.sh \
  -r git@github.com:gradle/ge-solutions.git \
  -p sample-projects/maven/3.8.x/ge \
  -g 'package'

image

Offline
./01-validate-local-build-caching-same-location.sh \
  -r git@github.com:gradle/ge-solutions.git \
  -p sample-projects/maven/3.8.x/ge \
  -g 'package' \
  -x

No visual differences.

image

Experiment 2

Online
./02-validate-local-build-caching-different-locations.sh \
  -r git@github.com:gradle/ge-solutions.git \
  -p sample-projects/maven/3.8.x/ge \
  -g 'package'

image

Offline
./02-validate-local-build-caching-different-locations.sh \
  -r git@github.com:gradle/ge-solutions.git \
  -p sample-projects/maven/3.8.x/ge \
  -g 'package' \
  -x

No visual differences.

image

Experiment 3

./03-validate-remote-build-caching-ci-ci.sh \
  -1 https://ge.solutions-team.gradle.com/s/nhrdhjwc6nysy \
  -2 https://ge.solutions-team.gradle.com/s/v7folxiq3dkyk

image

Experiment 4

./04-validate-remote-build-caching-ci-local.sh \
  -1 https://ge.solutions-team.gradle.com/s/ehiqsjd2wh5qs

image

@erichaagdev erichaagdev self-assigned this Dec 28, 2022
@runningcode
Copy link
Contributor

Took a very quick look and the direction looks good.

@erichaagdev erichaagdev marked this pull request as ready for review December 28, 2022 18:40
This was referenced Dec 29, 2022
Merged
Merged
Merged
erichaagdev
erichaagdev commented Dec 29, 2022
View reviewed changes
This was referenced Dec 30, 2022
Merged
Merged
Merged
Merged
Merged
@erichaagdev erichaagdev mentioned this pull request Jan 6, 2023
Merged
jthurne
jthurne approved these changes Jan 7, 2023
View reviewed changes
@runningcode runningcode removed their request for review February 28, 2023 11:38
@erichaagdev erichaagdev force-pushed the erichaagdev/display-realized-build-time-savings branch from bcadbb4 to c1d8a1a Compare March 6, 2023 00:03
@erichaagdev erichaagdev merged commit d4ca410 into main Mar 6, 2023
@erichaagdev erichaagdev deleted the erichaagdev/display-realized-build-time-savings branch March 6, 2023 15:57
etiennestuder
etiennestuder reviewed Mar 6, 2023
View reviewed changes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@etiennestuder etiennestuder etiennestuder left review comments

@jthurne jthurne jthurne approved these changes

Assignees

@erichaagdev erichaagdev

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@erichaagdev @runningcode @etiennestuder @jthurne